Skip to content

Conversation

@martindevans
Copy link
Member

@martindevans martindevans commented Oct 26, 2024

Updated to 958367bf530d943a902afa1ce1c342476098576b, built with https://github.com/SciSharp/LLamaSharp/actions/runs/11509292954

This is a much smaller update that the last one. Only notable changes are:

  • Some changes in the fill-in-middle (FIM) token functions (no change to C# API)
  • New FIM tokens exposed, not sure what these are for so no useful comments, if anyone knows a tip would be appreciated!
  • Removed softmax sampler. It's deprecated upstream, so it's best to remove it quickly on our end since that's a new API no one will be depending on yet.
  • Removed tail free sampler. Also deprecated upstream (llama : remove Tail-Free sampling ggml-org/llama.cpp#10071)

Testing:

  • Windows CPU
  • Windows CUDA
  • Windows Vulkan
  • Linux CPU
  • Linux CUDA
  • Linux Vulkan
  • MacOS CPU
  • MacOS Metal

@m0nsky
Copy link
Contributor

m0nsky commented Oct 26, 2024

Unit tests passed on Windows CUDA.

Test application is running fine on:

  • Windows CPU
  • Windows CUDA
  • Windows Vulkan
  • Linux CPU
  • Linux CUDA
  • Linux Vulkan

@martindevans martindevans requested a review from SignalRT November 1, 2024 21:20
@martindevans martindevans merged commit b321839 into SciSharp:master Nov 7, 2024
6 checks passed
@martindevans martindevans deleted the nov_binaries branch November 7, 2024 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants